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(54) Method and apparatus capable of transferring very high data rates across a midplane or 
backplane 



(57) A method and apparatus of communicating da- 
ta packets across the midplane of an electronic system 
in which the packets are partitioned into segments of a 
predetermined size and then serialized to a predeter- 
mined width. The serialized packets are transmitted, in 
phase staggered segments, across the midplane on a 



respective channel, received into receiving end and the 
serialized segments that have traversed the midplane, 
are deserialized and reassembled into the original data 
packet. 
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Description 

[0001 ] This invention relates to a bus capable of trans- 
ferring variable length packets (e.g. for POS) at a 
1 0Gbps rate between two separate cards across a mid- 
plane or backplane. 

[0002] The midplane or backplane (midplane/back- 
plane) has a limited number of physical pins available 
and signals must pass through two connectors, which 
potentially could introduce signal integrity issues for 
high-speed signals. 

[0003] There is a width vs. speed tradeoff, wherein a 
slower bus rate is easier and more reliable to implement 
however it must also be wider, which can be inefficient 
for short packets. 

[0004] SQULB (prior art) is a sequenced utopia-3 like 
bus designed for asynchronous transfer mode (ATM) 
applications, i.e. fixed sized (56-byte) cells, and as such 
is not capable of handling variable length packets. 
[0005] In order to modify SQULB to handle variable 
length packets, the bus must be made four times wider 
(four-bytes to sixteen-bytes). This solution is not feasi- 
ble for the following reasons: 

1 . The limited number of pins available across the 
midplane/backplane. There is currently no offering 
of serializer/deserializer (SERDES) devices capa- 
ble of handling this bus width. Separate SERDES 
devices would make it difficult to receive the packet 
data in the proper order and with minimal signal 
skew. 

2. A sixteen-byte wide bus implies that a single 
packet could contain up to fifteen empty bytes trans- 
ferred during the end of packet. This transmission 
model would be very inefficient and would require 
a large increase in the operating frequency of the 
physical bus to maintain a 10Gbps rate. 

[0006] The present invention provides a method of 
communicating data packets across a midplane or a 
backplane of an electronic system, solving the problem 
that will be encountered by any efforts to pass variable- 
length packet data between separate cards. 
[0007] More precisely, the present invention provides 
a method of communicating data packets across a mid- 
plane or a backplane of an electronic system, compris- 
ing the steps of: 

a) forming a plurality of channels through said mid- 
plane or backplane, 

b) partitioning each data packet into segments of a 
predetermined size; 

c) serializing each segment to a predetermined 
width; 

d) transmitting each serialized segment across said 
midplane or backplane on a respective channel, 

e) receiving, from their respective channels, serial- 
ized segments that have traversed the midplane or 



backplane; 

f) deserializing the received serialized segments; 
and 

g) re-assembling the deserialized segments into the 
5 data packet. 

[0008] The present invention is directed to a method 
for partitioning packets into segments of a predeter- 
mined size (e.g. 64 bytes), serializing the segments, and 

10 transmitting the segments over a plurality of channels 
(for example, four) that have a staggered phase rela- 
tionship to one another, and wherein the phase differ- 
ence between adjacent channels (actually, adjacent 
segments in the sequence of segments that form the 

15 packet) is more than the maximum latency that can oc- 
cur in any one channel, thereby maintaining the ordering 
of the serialized packet segments. Preferably, there are 
64-byte segments divided into four channels and stag- 
gering is 2 times maximum latency. While the invention 

20 is particularly applicable to variable length packets, the 
invention can also be used for fixed-length cells as well 
as variable length packets. 

[0009] The advantages of the invention, in general, 
and over the closest prior art solution include: 

25 

1 . Variable length packets can be transmitted very 
efficiently at a low bus frequency while maintaining 
a bandwidth of 1 0Gbps. Furthermore, in the present 
implementation, the segmentation of packets into 

30 64-byte segments means that ATM cells can also 
be transmitted very efficiently over the bus. 

2. The described bus topology is easily amenable 
to quad OC48. 

3. Since the bus is divided into segments transmit- 
35 ted four-bytes at a time, there is only a need for a 

maximum of three empty bytes per packet. This 
scheme makes the bus more efficient and allows it 
to operate at a slower frequency while still achieving 
10Gbps rates. 

40 

[0010] The above and other objects, advantages and 
features of the invention will become more apparent 
when considered with the following specification and ac- 
companying drawings wherein: 

45 

Figure 1 is a block diagram illustrating the packets 
divided into 64-byte segments for transmitting over 
four channels are staggered in their phase relation- 
ship to one another, 
so Figure 2 is a timing chart showing a single channel 
bus timing of a given example, and 
Figure 3 illustrates the staggering performed by the 
transmitting device. 

55 [0011 ] Referring to Figure 1 , packets from transmitter 
T are divided into 64-byte segments for transmission 
over four channels CHA, CHB, CHC and CHD that are 
staggered in their phase relationship to one another. A 
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first array or set of SERDES (serializer/deserializer) de- 
vices TSA, TSB, TSC and TSD at the transmitter T in 
each channel is used to serialize the data and control 
for transmission across the midp lane/backplane P and 
a second array or set of SERDES devices RSA, RSB, 
RSC and RSD are used at the receiver R in each chan- 
nel to deserialize the data. These SERDES devices al- 
low data and control to be passed across the midplane/ 
backplane in acompressed manner and thus reduce the 
large variable amount of latency for a complete packet 
transfer. Furthermore, by staggering the phase relation- 
ship of the channels, the maximum latency that can oc- 
cur in any one channel is accommodated and therefore 
the ordering of packet segments is maintained. For ex- 
ample, in the embodiment implemented in Figure 1 , the 
channels are staggered by 40 ns and the maximum la- 
tency per channel is 20 ns. 

[001 2] The invention has two basic aspects: the topol- 
ogy of Figure 1 and a bus protocol that runs on that to- 
pology to provide the required bandwidth of 1 0Gbps for 
variable length packets. 

[0013] With reference to Figure 2, the bus protocol will 
now be briefly described: 

Egress Data Path Signal Definitions 

[0014] The following describes a single channel and 
what is described is applicable for all four channels and 
can easily be extended to the complete bus. 
[0015] Each channel includes a plurality of core data 
path signals, a 32-bit wide data bus with eleven-bits of 
out-of-band control, and a number of non-core data path 
signals, which may be used to transfer additional infor- 
mation. 

[0016] Packets are broken apart into segments, 
64-bytes of data, and transmitted 32-bits per cycle over 
1 6 clock cycles. Packets that are greater than 64 bytes 
are required to be transmitted over more than one chan- 
nel. 

[001 7] When data is available for transmission, a Start 
of Segment (SOS) is raised for one clock cycle concur- 
rent with the first word of the transfer. 
[0018] When a new packet is being transmitted, a 
Start of Packet (SOP) signal is raised for one clock cycle 
concurrent with the first word of the packet. Similarly, 
when the last word of a packet is being transmitted, an 
End of a Packet (EOP) signal is raised for one clock cy- 
cle concurrently with the last word. The Empty (MPTY) 
signals indicate how many bytes of the current word are 
valid. Since the bus is a word-wide (i.e. 32-bits will be 
transferred each clock cycle) up to three bytes of PAD 
may be present on a transfer. The MPTY signals are 
only valid when an EOP occurs. If the packet happens 
contain an error, then the Error (ERR) signal becomes 
active while EOP is active. 

[0019] During the complete data transfer, the Valid 
(VAL) signal is active. The parity across this interface 
should always be valid for both the data path (DPRTY) 



signal and the control path (CPRTY) signal. 
[0020] If there is no data to be transferred, idle cycles 
will occur on the bus. For an idle cycle, the data bus and 
all the control signals (except parity) will be driven low. 
5 The parity will remain valid at all times. 

Summarizing: 

[0021] 

10 

1 . Data packets are divided into 64-byte segments, 

2. This interface has parity protection for all data 
and control signals (DPRTY and CPRTY), 

3. Each segment is serialized to four bytes wide 
15 [Data(31 :0)] and each segment is tagged with a set 

of out-of-band control signals, as shown in Figure 

2. The associated control signals consist of start 
and end of packet indications (SOP and EOP), error 
indication (ERR), and the number of empty bytes 

20 transmitted during the end of packet condition. This 
information is then used by the receiver to properly 
assemble the packet. 

[0022] Figure 3 illustrates the staggering performed 
25 by the transmitting device T. 

[0023] The advantages of the invention, in general, 
and over the closest prior art solution include: 

1 . Variable length packets can be transmitted very 
30 efficiently at a low bus frequency while maintaining 
a bandwidth of "lOGbps. Furthermore, in the present 
implementation, the segmentation of packets into 
64-byte segments means that ATM cells can also 
be transmitted very efficiently over the bus. 
35 2. The described bus topology is easily amenable 
to quad OC48. 

3. Since the bus is divided into segments transmit- 
ted four-bytes at a time, there is only a need for a 
maximum of three empty bytes per packet. This 

40 scheme makes the bus more efficient and allows it 
to operate at a slower frequency while still achieving 
1 0Gbps rates. 

[0024] The invention is directed to the method and ap- 
45 paratus of partitioning packets into segments of prede- 
termined size (for example, 64-bytes), serializing the 
segments, and transmitting them over a plurality of 
channels (for example, four) that have a staggered 
phase relationship to one another, and wherein the 
so phase difference between adjacent channels (actually, 
adjacent segments in a sequence of segments that form 
the packet) and more than the maximum latency that 
can occur in any one channel; thereby maintaining the 
order of the serialized packet segments. The invention 
55 is not limited to variable-length packets as the invention 
can be used for both fixed-length cells and variable- 
length packets. 

[0025] This invention addresses an obstacle and 
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solves the problem that will be encountered by any ef- 
forts to pass variable-length packet data between sep- 
arate cards. 

[0026] While the invention has been described in re- 
lation to preferred embodiments of the invention, it will 
be appreciated that other embodiments, adaptations 
and modifications of the invention will be apparent to 
those skilled in the art. 



Claims 

1. Method of communicating data packets across a 
midplane or a backplane of an electronic system, 
comprising the steps of: 15 

a) forming a plurality of channels through said 
midplane or backplane, 

b) partitioning each data packet into segments 

of a predetermined size; 20 

c) serializing each segmentto a predetermined 
width; 

d) transmitting each serialized segment across 
said midplane or backplane on a respective 
channel, 25 

e) receiving, from their respective channels : se- 
rialized segments that have traversed the mid- 
plane or backplane; 

f) deserializing the received serialized seg- 
ments; and 30 

g) re-assembling the deserialized segments in- 
to the data packet. 

2. Method defined in claim 1 wherein said step d) in- 
cludes the step of delaying the transmission of a 35 
segment in one channel from that of a channel car- 
rying a next sequential segment of the data packet 

so that the segments are staggered relative to each 
other. 

40 

3. Method defined in claim 1 or claim 2 wherein step 
d) further includes tagging the packet with the out- 
of-band control and error correction/detection sig- 
nals. 

45 

4. Apparatus in a communication system having a dig- 
ital data transmitter side apparatus for transmitting 
digital data packets of variable length between two 
separate points across a midplane or a backplane 

to a digital data receiver side apparatus, said mid- 50 
plane or backplane having a limited number of 
physical pins available, said apparatus comprising: 

means at said digital data transmitter for parti- 
tioning each digital data packet into segments 55 
of a predetermined size, 
an array of serializers connected to said trans- 
mitter side for serializing each segment into a 



predetermined width, each serializer being 
connected to a separate pathway through said 
midplane or backplane, 

an array of deserializers on said receiver side 
corresponding in number to the number of se- 
rializers on said transmitter side of said mid- 
plane or backplane and corresponding thereto, 
respectively, for receiving from their respective 
channels the serialized segments of data tra- 
versing said midplane or backplane and means 
for reassembling the deserialized segments in- 
to the data packet. 

5. Apparatus defined in claim 4 wherein in said parti- 
tioning of packets into a predetermined size and se- 
rializing of segments thereof and transmitting said 
segments over a plurality of channels, said chan- 
nels have a staggered phase relationship to one an- 
other wherein the phase difference between adja- 
cent channels is more than the maximum latency 
that can occur in any one channel thereby maintain- 
ing the ordering of the serialized packet segments. 

6. Apparatus defined in claim 5 wherein said data seg- 
ments are in 64-byte segments and the number of 
channels in said midplane are four and staggering 
is two times the maximum latency. 

7. Apparatus defined in any one of claims 4 to 6 where- 
in said transmission of segments in the respective 
channels are delayed from that of a channel carry- 
ing a next sequential segment of the packet. 

8. Apparatus defined in any one of claims 4 to 7 in- 
cluding means for tagging the packets with an out- 
of-band control and error correction/detection sig- 
nals. 

9. Communication system having a digital data trans- 
mitter side apparatus for transmitting digital data 
packets of variable length between two separate 
cards across a midplane or a backplane to a digital 
data receiver side apparatus, said transmitter side 
apparatus including means for partitioning each 
digital data packet into segments of a predeter- 
mined size and an array of serializers connected to 
said transmitter side for serializing each segment 
into a predetermined width, each serializer being 
connected to a separate pathway through said mid- 
plane or backplane. 

10. Receiver side apparatus in a communication sys- 
tem having a digital data transmitter side for trans- 
mitting digital data packets of variable length be- 
tween two separate cards across a midplane or a 
backplane to a digital data receiver side apparatus 
and said transmitter side apparatus forming multi- 
ple channels, 
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said receiver side apparatus being comprised 
of an array of deserializers corresponding in 
number to the number of serializes on said trans- 
mitter side for receiving from their respective chan- 
nels the serialized segments of data that traverse 5 
said midplane and reassembling the deserialized 
segments of data into data packets. 
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